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The Kilander Health Knowledge Test is analyzed, based 
on responses of four independent groups, to determine if response 
tendencies to test items were peculiar to groups studied and if test 
items retained independence regardless of the population group. .The 
1969 edition of the Kilander Health Knowledge Test was administered 
to all ninth- and twelfth-grade students attending a high school 
representing both a high and low socioeconomic level. The test was 
deemed to be sensitive to response tendencies of independent 
population groups. Analysis procedures also established that test 
item categories were independent regardless of the population. 
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ITEM ANALYSIS OF KILANDBtt HEALTH KNOWLEDGE 
TEST USING FOUR POPULATION GROUPS 



In a series of provocative papers, Berg (1, 2) has insisted 
that individuals taking a test make many responses which are simi- 
lar to responses made by others, but they also make responses 
which are unlike those of others in the general population. Res- 
ponses in the second category were reported to be most like those 
made by members of some special sub-population to which that parti- 
cular individual belonged. Berg concluded that test content is 
secondary to clearly delineated criterion groups and an analysis 
of response tendencies peculiar to the particular groups* 

Investigators have reported the use of the Ki lander Health 
Knowledge Test as the criterion instrument to assess health know- 
ledge of various population groups. In spite of the fact that 
significant difference in group means has been reported, little 
evidence has been presented to establish that the difference in 
mean scores represent a difference in knowledge level and not a 
difference in response tendencies peculiar to sub-populations 
studied. 

The purpose or this investigation was to conduct an item 
analysis of the Kilander Health Knowledge Test based on the res- 
ponses of four independent population groups in order to determine 
if response tendencies to test items are peculiar to groups studies 
and if test items retain item independence regardless of the 
population group. 



METHODOLOGY 

Criterion Instrument and Measures 

The 1969 edition of the Kilander Health Knowledge Test was used 
as the criterion instrument and the criterion measures consisted of 
the total correct response score, nine category scores, and item 
response to each of the 100 questions. 

This test was designed for high school and college students 
and consists ot 100 multiple choice questions. Questions were 
selected from nine aspects of health education; however, a few 
items test attitudes rather than knowledge. Reliability of .80 
for college freshmen and .83 for high school seniors has been 
reported. Scores ranging from 22 to 82 and a mean of 56.15 have 
been reported for high school seniors. Several investigators have 
used the instrument with high school populations (5, 6, 7, 10, 12); 
however, the instrument has more often been used with college popu- 
lations (3, 4, 8, 12, 13, 14, 15). Test item discrimination has 
not been reported. 

Criterion Groups 

During the fall of 1971 the criterion instrument was adminis- 
tered to all of the ninth and twelfth grade students attending two 
senior high schools in the City of Portland school district. The 
designation of the two schools was made by the central administra- 
tion and approved by the principal of the school involved. The 
criterion for the selection of participating schools specified 
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that the student body of a school would represent a high socio- 
economic level or a low socioeconomic level but would otherwise 
reflect similar curriculum opportunities, an equal balance in 
racial and ethnic population, a comparable student-teacher ratio, 
and proportional size. Socioeconomic level was to be determined 
by estimated average income, occupation of parents, and value of 
residence (9, 16) • The two selected schools represented a reason- 
ably clear delineation of the criterion. The literature suggested 
that difference in response might be expected by grade level as 
well as socioeconomic level (5, 6). Thus four criterion groups 
were established to include a low socioeconomic level ninth grade 
(LSL-9), low socioeconomic level twelfth grade (LSL-12), high socio- 
economic level ninth grade (HSL-9), and high socioeconomic level 
twelfth grade (HSL-12). A significant P value between socio- 
economic level (F=139.56) and between grade levels (F=195.12) has 
been reported as has a significant interaction effect (F=5.74) (6). 
Therefore, the four groups were regarded as four independent popula- 
tion groups, 

% 

Test Administration and Scoring 

The standard procedures for the administration of the Kilander 
test were precisely executed, except that each student recorded the 
response to each of the 100 items on an IBM 503 answer sheet instead 
of marking the test booklet. The test was administered to the 
ninth grade students by the ninth grade English teachers and to the 
twelfth grade students by the twelfth grade social science teachers. 



Each classroom teacher was assisted by a member of the research 
team. The investigators met with each classroom teacher prior to 
the administration of the test to standardize testing procedures 
and to distribute test materials (6). 

The completed answer sheets were processed by an IBM 1232 
Optical Mark Page Reader and a 534 Card Punch to provide punched 
data tab cards for each subject 1 s responses. These cards were sub- 
sequently submitted to a CDC 3300 computer with a program to obtain 
punched data cards containing the total and the category scores. 
These data cards were used to obtain distribution statistics 9 item 
analysis data to include Difficulty Index and Discriminant Index, 
distribution of responses to include X values for independence of 
distribution, and correlation coefficients of the criterion scores, 

RESULTS AND DISCUSSION 
The mean score and distribution statistics for the responses 
on the Kilander Health Knowledge Test by four independent popula- 
tion groups are presented in Table 1. Scores for each population 
group were arranged in order of magnitude. Respondents with scores 
in the upper and lower 27 percent of the range were selected from 
computation of the Discriminant Index and Difficulty Index (11). 
The number of respondents in the lower 27 percent of the range of 
scores represented an equal proportion of each criterion group. 
The number in the upper 27 percent of the range of scores also 
represented an equal proportion with the exception of the LSL-9 
group which had a smaller proportion of the total group in the 
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upper 27 percent. A comparison of the range of scores, median, 
mean, and standard deviation would suggest that the socres for the 
LSL-9 group were skewed to the low side. 

Distribution of responses, percent of correct responses for 
the total group, and for both the upper and lower sub-groups, 
Difficulty Index, and Discriminant Index were obtained for each 
of the 100 items for each of the four independent groups. The 
results of item number 9, which had the highest Difficulty Index, 
and item number 91, which had the lowest Difficulty Index are 
presented in Table 2. Both items had a significant X 2 vtlue for 
independence of distribution of responses. Neither item had a 
significant X value for independence when the two twelfth grade 
groups were considered separately. 

The difference between the means for the LSL-9 and LSL-12 was 
12.89, while th~ Jifference between HSL-9 and HSL-12 was 9.38. 
This would sugeest that the students in the LSL school gained more 
information between the ninth and twelfth grades than did the stu- 
dents in the HSL school. Students in the LSL-12 scored only 
slightly higher than did the HSL-9 students. 

Test item 9 dealt with the specialization of physicians. The 
students in the LSL school had only a change of 2 points in diffi- 
culty between the ninth and twelfth grades while the students in 
the HSL school had a change of 17 points between the ninth and 
twelfth grades. This would indicate that the students in the 
HSL group gained more knowledge about specialization of physicians 
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than did the students in the LSL group* Test item 91 dealt with 
mental disorders. The low Difficulty Index would indicate that 
this was a difficult question for all students. 

The ten least difficult and the ten most difficult items on the 
Kilander Health Knowledge Test as determined by the Difficulty Index 
are reported in Table 3. Difficulty Index was determined on the 
bases of the percent of correct answers by respondents in the upper 
27 percent of the range as contrasted to percent of correct answers 
by respondents in the loner 27 percsnt as both were contrasted to 
the percent of correct replies by the total group. A high index 
score established items which were answered correctly by roost res- 
pondents while a low index score indicated a tendency to hav? an 
incorrect response • 

Items 9, 29 and 55 (Table 3) were included by all four criter- 
ion groups when the ten items with the highest Difficulty Index 
were selected for each of the four groups. These items represented 
the least difficult items. The remaining seven items in Table 3 
represented items which had the highest averaged Difficulty Index 
score across four criterion groups. Some of these items were not 
included in the ten least difficult items for the specific group. 

Items 91, 63, 97 and 88 in the Low Difficulty Index column of 
Table 3 were items to have a common low Difficulty Index when the 
ten items with the lowest difficulty index were selected for each 
criterion group. These four items represented the most difficult 
items for all respondents. The remaining six items represented 
questions which had the lowest averaged Difficulty Index score 
q across the four groups. 

ERIC 



-7- 



In addition to the ten items shown in Table 3, items 14, 71, 
BO and 85 were particularly difficult for the LSL-9 group, while 
items 4, 5, 94 and 95 were additional difficult questions for the 
HSL-9 group. The LSL-12 group seemed to have difficulty with items 
4, 11 and B3 while the HSL-12 group had items 66 and 83 in the ten 
most difficult list. 

Items with considerable variation in Difficulty Index by the 
four population groups are presented in Table 4. The distribu- 
tion of responses by population groups, X for independence of dis- 
tribution of responses, and correct response are also presented. 
Responses to these ten items seemed to account for much of the 
difference in groups means. 

Analyses of the test items found in both the high Difficulty 
Index and low Difficulty Index indicated that no consistency in 
health topics existed in either group. The difficulty of the test 
item appeared to be within the item rather than within the health 
topic, Chi square values for these items established that all but 
item 45 and 63 had independence of distribution of responses. 
Test items 50, 69, 73, 83 and 94 had an increase in difficulty from 
the ninth to the twelfth grade. Since each ot these items per- 
tained to different health topics, the increase in difficulty would 
seem to be due to the item rather than to a deficiency in knowledge 
related to specific health topics. 

Four items with a common high Discriminant Index and four items 
with a common low Discriminant Index are presented in Table 5. 
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Discriminant Index is the difference between the percent of correct 
answers for respondents in the upper 27 percent of the range of scores 
and the percent of correct answers for the respondents in the lower 
27 percent of the range. The Index represented the difference in the 
extent to which the upper 27 percent consistently marked the item 
correct and the lower 27 percent consistently marked the item 
correct* A high Discriminant Index would indicate that the upper 
group had a high percent of correct response and the lower group 
had a low percent of correct response. A low Discriminant Index 
would indicate that the difference between the extent to which the 
upper group gave the correcc response and the lower group gave the 
correct response was minimal, A negative index score would indicate 
that the respondents in the lower 27 percent of the range of scores 
gave the correct response more consistently than those in the upper 
27 percent of the range of scores* 

Items 35, 49, 54 and 61 had high Discriminant Index for ninth 
graders but not twelfth graders, while items 14, 48, 80 and 89 had 
high Discriminant Index for twelfth graders but not ninth graders. 
The LSL groups had high Discriminant Index on item 42 and the HSL 
groups had a high Discriminant Index on item 100. Item 95 had 
a common low Discriminant Index for both ninth grade groups but 
not twelfth grade groups. In addition, to the four items shown in 
Table 5, the LSL-9 group had 16 items with low Discriminant Index, 
while LSL-12 had only 4, HSL-9 only 5 and HSL-12 only 2 other items. 
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Items 94 and 96 hadiA*corem<fen negative Discriminant Index across 

'. <SJb 

all groups* The 1~SL~9 group had six additional items (45, 53, 

65, 81, 90 and 93) with a negative Discriminant Index. The KSL-12 

group had a negative Discriminant Index on items 6B an/ 93. 

The Discriminant Index indicated very little difference bet- 
ween the two high schools. The ninth grade group in the LSL 
school had more items with a low Discriminant Index and more with 
a negative index than any of the other groups. This would indicate 
that for those items the students in the ninth grade of the LSL 
high school had less knowledge than the other students* The 
twelfth grade students in the same school had considerably fewer 
items with a low or negative Discriminant Index. These results 
suggest that between the ninth and twelfth grades the students 
in the LSL high school gained more health knowledge than did the 
students in the HSL high school. 

Chi square analysis ot the distribution of the four possible 
responses for each of the four groups for each of the 100 test items 
was computed from a 4 by 4 contingency table in order to determine 
if an equal number of respondents would have the same distribution. 
Independence values were first computed for the expected frequences. 
These independence values were obtained by multiplying the fre- 
quences of the rows and columns for a given cell and dividing this 
value by N. The chi square for each cell was then computed in the 
usual manner. Chi square necessary for significance with 9 df at 
.01 level was 21.66. Items 5, 8, 12, 15, 16, 25, 27, 45, 55 and 63 

O 
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were the only items for which .a significant X** value for ind^Kndence 
of distribution was not obtained. Five of these itens were from the 
Personal Health category of the test and the others were distributed 
across the remaining categories. 

Correlation of category scores to total score was obtained 
for each of the four population groups for each of the nine cate- 
gories of the test (Table 6). While the significance of the dif- 
ference between correlation coefficient values for the four indepen- 
dent samples was not obtained, no consistent pattern of difference 
or jr for category to total scores was apparent. Regression coe- 
fficient of the category to total score was also computed with the 
results shown in Table 6. Perusal of the regression coefficient 
across criterion groups will suggest common values for each depen- 
dent measurement. 

While the Discriminant Index, Difficulty Index and X values 
for independence of response would suggest that the four populations 
did respond to items with some degree of independence, the consis- 
tency of the correlation coefficient and regression coefficient 
values suggested that the test items had common response tenden- 
cies within t..e population groups. 

Using a full selection of response scores, a biseriai coef- 
ficient correlation item analysis procedure was used to compare 
the distribution of item scores to the total score as well as the 
item score to the category score for all of the items in the 
Family Living, Mental Health, and Safety Education categories. 



On the basis of a multivariate discriminate analysis, response 
scores on these three categories had been reported to be the primary 
contributors to the separation of the four population groups (6). 
While no consistent pattern of difference was apparent, a consider* 
able difference in £^£ s values will be observed. 

Within each category an item-to-item correlation coefficient was 
also obtained. For the nine Family Living category items, a low 
item-to-item correlation was found for ail items for the four 
criterion groups with the exception of items 61 to 64, 61 to 74, and 
64 to 74 for the LSL-9 group. For the five Mental Health items, a 
low £ for all items was obtained with the exception of items SO to 
56 for both the ninth and twelfth grade LSL groups. Four the three 
Safety Education items a low £ for ail items across four indepen- 
dent groups was obtained. These results would suggest that with 
the possible exceptions cited that the test items within the cate- 
gory were independent. 

SUMMARY AND CONCLUSION 
On the basis of the item analysis procedures and population 
groups used in this investigation, the Kilander Health Test was 
deemed to be sensitive to response tendencies of independent popula- 
tion groups and at the same time to have a high degree of consis- 
tency within the population groups. Analysis procedures also 
established that test items within categories were independent 
regardless of the population. 
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ANALYSIS OF KI LANDER HEALTH KNOWLEDGE TEST ITEMS WITH 
HIGHEST AND LOWEST DIFFICULTY INDEX BY FOUR GROUPS 
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TABLB 4 



DISTRIBUTION OF RESPONSES ON TEN K I LAND BR HEALTH KNOWLEDGE 
TEST ITEMS WITH CONSIDERABLE VARIATION IN 
DIFFICULTY INDEX ACROSS FOUR GROUPS 
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TAttLE 6. 

CORRELATION COEFFICIENTS AND REGRJSSION COEFFICIENTS 
OF TOTAL SCORE TO NINE CATEGORY SCORES 
BY FOUR CRITER TON GROUPS 
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